Data Compression in Database Query Processing

Row-oriented databases (or “row-store”) employ data compression methods (like dictionary encoding) to reduce the I/O cost by decreasing the data sizes. However, there are two limitations on row-stores when applying data compression schemes: (1) row-stores only allow encoding one single value at a time, and (2) they have to pay the decompression cost in query processing. The above shortcomings limit the wide usage of data compression in row-oriented databases. On the contrary, column-oriented databases (or “columnstore”) provide more opportunities for data compression as the values of the same attribute are stored consecutively. In a column-oriented database, compression schemes that encode multiple values crossing multiple rows at once are allowed, but such schemes do not work in row-stores. In addition, column-stores can sometimes perform queries directly on compressed data without decompression, which yields the ultimate performance boost, since the I/O cost is saved by accessing less data and the decompression cost is also avoided. However, column-stores do not consider heavyweighted compressions since they can not support random accesses, though they have high compression qualities. This research exam report surveys different compression techniques on both row-stores and column-stores. For each compression scheme, we first review the existing work, then present our ideas. In addition, we introduce a class of queries that both row-stores and column-stores miss key opportunities to answer them efficiently. Then we present a new data structure, which allows both light-weighted and heavyweighted data compressions, to answer the queries efficiently.